Skip to content

test(vdsnapshot): disable filesystem frozen check#981

Closed
eofff wants to merge 3 commits into
mainfrom
test/vdsnapshot/disable-fs-frozen-check
Closed

test(vdsnapshot): disable filesystem frozen check#981
eofff wants to merge 3 commits into
mainfrom
test/vdsnapshot/disable-fs-frozen-check

Conversation

@eofff
Copy link
Copy Markdown
Contributor

@eofff eofff commented Apr 28, 2025

Description

It is a known issue that filesystem not always freeze while vd snapshot tests.
To prevent this error from causing noise during testing, we disabled this check.
It will need to be re-enabled once this issue is fixed.

Why do we need it, and what problem does it solve?

What is the expected result?

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: vdsnapshot
type: chore
summary: disable filesystem frozen check
impact_level: low

@eofff eofff requested a review from Isteb4k April 28, 2025 07:54
@eofff eofff self-assigned this Apr 28, 2025
@Isteb4k Isteb4k added this to the v0.18.0 milestone Apr 28, 2025
@eofff eofff force-pushed the test/vdsnapshot/disable-fs-frozen-check branch 2 times, most recently from 9aefeb1 to b320205 Compare April 30, 2025 15:55
@Isteb4k Isteb4k marked this pull request as draft May 6, 2025 13:17
@universal-itengineer universal-itengineer modified the milestones: v0.18.0, v0.19.0 May 12, 2025
@eofff eofff closed this May 13, 2025
@eofff eofff force-pushed the test/vdsnapshot/disable-fs-frozen-check branch from b320205 to b6ced4e Compare May 13, 2025 13:45
@eofff eofff reopened this May 14, 2025
@eofff eofff marked this pull request as ready for review May 14, 2025 07:52
Comment thread tests/e2e/vd_snapshots_test.go Outdated
watcher, err := virtClient.VirtualMachines(conf.Namespace).Watch(context.Background(), v1.ListOptions{TimeoutSeconds: ptr.To(int64(filesystemReadyTimeout.Seconds()))})
Expect(err).NotTo(HaveOccurred())
var hasFrozen bool
for {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need timeout

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for {
  select {
  case <-time.After(5 * time.Minute):
    panic ....
  }
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already have timeout in ListOptions

Comment thread tests/e2e/vd_snapshots_test.go Outdated

watcher, err := virtClient.VirtualMachines(conf.Namespace).Watch(context.Background(), v1.ListOptions{TimeoutSeconds: ptr.To(int64(filesystemReadyTimeout.Seconds()))})
Expect(err).NotTo(HaveOccurred())
var hasFrozen bool
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

declare in for

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After fix problem, we will use it state outer for

Comment thread tests/e2e/vd_snapshots_test.go Outdated
break
}

vm, ok := event.Object.(*virtv2.VirtualMachine)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it old or new object?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eofff eofff requested a review from yaroslavborbat May 14, 2025 09:46
Comment thread tests/e2e/vd_snapshots_test.go Outdated
Comment thread tests/e2e/vd_snapshots_test.go Outdated
Signed-off-by: Valeriy Khorunzhin <valeriy.khorunzhin@flant.com>
@eofff eofff force-pushed the test/vdsnapshot/disable-fs-frozen-check branch from 464536b to deaa7aa Compare May 22, 2025 11:05
Signed-off-by: Valeriy Khorunzhin <valeriy.khorunzhin@flant.com>
@eofff eofff requested a review from Isteb4k May 22, 2025 12:04
Copy link
Copy Markdown
Contributor

@hardcoretime hardcoretime left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should uncomment this check because the test does not check that the snapshot becomes ready.

// By(fmt.Sprintf("Snapshots should be in %s phase", PhaseReady))

Signed-off-by: Valeriy Khorunzhin <valeriy.khorunzhin@flant.com>
@eofff eofff added the e2e/run Run e2e test on cluster of PR author label May 23, 2025
@deckhouse-BOaTswain
Copy link
Copy Markdown
Contributor

deckhouse-BOaTswain commented May 23, 2025

Workflow has started.
Follow the progress here: Workflow Run

The target step completed with status: success.

@deckhouse-BOaTswain deckhouse-BOaTswain removed the e2e/run Run e2e test on cluster of PR author label May 23, 2025
for {
hasFrozen = false
for _, frozen := range vmFSFrozenByName {
hasFrozen = hasFrozen || frozen
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hasFrozen is always false here; you unconditionally set it to false above

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you need to iterate all the virtual machines if you've already found the frozen one? Why not just use a break for the for loop at line 249?


vm, ok := event.Object.(*virtv2.VirtualMachine)
if !ok {
continue
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will result in an infinite loop in case of an error. If something goes wrong, throw an error in the test

}
_, err := WaitForVMsUnfrozen(int64(filesystemReadyTimeout.Seconds()))
Expect(err).NotTo(HaveOccurred())
// TODO: It is a known issue that VM filesystems after snapshot not unfreezing. To prevent this error from causing noise during testing, we disabled this check. It will need to be re-enabled once this issue is fixed.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

t seems like the issue has been fixed, it doesn't throw anymore. Let's restore the check.

filesystemReadyPollingInterval,
).Should(Succeed())
}
_, err := WaitForVMsUnfrozen(int64(filesystemReadyTimeout.Seconds()))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In fact, it's much more important for us to check the freezing event during the snapshot process rather than the unfreezing at the end of the test through Watch.

@Isteb4k Isteb4k marked this pull request as draft May 23, 2025 14:18
@Isteb4k Isteb4k modified the milestones: v0.19.0, v0.20.0 May 29, 2025
@nevermarine nevermarine modified the milestones: v0.20.0, v0.21.0 Jun 17, 2025
@nevermarine nevermarine modified the milestones: v0.21.0, v0.22.0 Jun 26, 2025
@universal-itengineer universal-itengineer modified the milestones: v0.22.0, v0.23.0 Jul 10, 2025
@nevermarine nevermarine modified the milestones: v0.23.0, v0.24.0 Jul 24, 2025
@universal-itengineer universal-itengineer modified the milestones: v0.24.0, v0.25.0 Aug 8, 2025
@nevermarine nevermarine modified the milestones: v0.25.0, v0.26.0 Aug 29, 2025
@nevermarine nevermarine modified the milestones: v1.0.0, v1.1.0 Sep 11, 2025
@eofff eofff closed this Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants